Systems and Methods for Position-Based Haptic Effects

ABSTRACT

One illustrative system disclosed herein includes a sensor configured to detect a gesture and transmit an associated sensor signal. The gesture includes a first position at a distance from a surface and a second position contacting the surface. The system also includes a processor in communication with the sensor and configured to: receive the sensor signal from the sensor, and determine one or more haptic effects based at least in part on the sensor signal. The one or more haptic effects are configured to provide substantially continuous haptic feedback throughout the gesture. The processor is also configured to generate one or more haptic signals based at least in part on the one or more haptic effects, and transmit the one or more haptic signals. The system includes a haptic output device for receiving the one or more haptic signals and outputting the one or more haptic effects.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 15/974,236, filed May 8, 2018, and entitled“Systems and Methods for Position-Based Haptic Effects,” which is acontinuation of U.S. patent application Ser. No. 14/966,652, filed onDec. 11, 2015, and now U.S. Pat. No. 9,990,078, issued on Jun. 5, 2018,and entitled “Systems and Methods for Position-Based Haptic Effects,”the entirety of each of which is hereby incorporated by referenceherein.

FIELD OF THE INVENTION

The present invention relates to the field of user interface devices.More specifically, the present invention relates to position-basedhaptic effects.

BACKGROUND

The quality of the interfaces through which humans interact withcomputer-based systems is becoming increasingly important. To createmore intuitive and enhanced user experiences, such systems may usevisual, audio, and/or haptic feedback to reproduce aspects ofinteractions in the physical world. Such feedback is often generatedupon a user contacting a touchscreen display or another user interfacecontrol. For example, a computer-based system may generate hapticfeedback upon a user contacting a virtual button on a touchscreendisplay. Contact-based haptic feedback, however, may not accuratelysimulate an interaction with a real-world object. Thus, there is a needfor additional systems and methods for providing haptic feedback.

SUMMARY

Embodiments of the present disclosure comprise computing devicesconfigured to generate position-based haptic effects. In one embodiment,a system of the present disclosure may comprise a sensor configured todetect a gesture and transmit a sensor signal associated with thegesture. The gesture may comprise at least two positions, a firstposition of the at least two positions comprising a distance from asurface and a second position of the at least two positions comprising acontact with the surface. The system may also comprise a processor incommunication with the sensor. The processor may be configured toreceive the sensor signal from the sensor. The processor may also beconfigured to determine one or more haptic effects based at least inpart on the sensor signal. The one or more haptic effects may beconfigured to provide substantially continuous haptic feedbackthroughout the gesture. The processor may further be configured togenerate one or more haptic signals based at least in part on the one ormore haptic effects, and transmit the one or more haptic signals. Thesystem may further comprise a haptic output device in communication withthe processor. The haptic output device may be configured to receive theone or more haptic signals and output the one or more haptic effects.

In another embodiment, a method of the present disclosure may comprise:receiving a sensor signal associated with a gesture from a sensor,wherein the gesture comprises at least two positions, a first positionof the at least two positions comprising a distance from a surface and asecond position of the at least two positions comprising a contact withthe surface. The method may also comprise determining one or more hapticeffects based at least in part on the sensor signal. The one or morehaptic effects configured to provide substantially continuous hapticfeedback throughout the gesture. The method may still further comprisegenerating one or more haptic signals based at least in part on the oneor more haptic effects, and transmitting the one or more haptic signalsto a haptic output device. The haptic output device may be configured toreceive the one or more haptic signals and output the one or more hapticeffects. Yet another embodiment comprises a computer-readable medium forimplementing such a method.

These illustrative embodiments are mentioned not to limit or define thelimits of the present subject matter, but to provide examples to aidunderstanding thereof. Illustrative embodiments are discussed in theDetailed Description, and further description is provided there.Advantages offered by various embodiments may be further understood byexamining this specification and/or by practicing one or moreembodiments of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

A full and enabling disclosure is set forth more particularly in theremainder of the specification. The specification makes reference to thefollowing appended figures.

FIG. 1A shows an embodiment of a system for providing position-basedhaptic effects;

FIG. 1B shows another embodiment of a system for providingposition-based haptic effects;

FIG. 2 is a block diagram showing a system for providing position-basedhaptic effects;

FIG. 3 shows an embodiment of a system for providing position-basedhaptic effects;

FIG. 4 shows another embodiment of a system for providing position-basedhaptic effects;

FIG. 5 shows still another embodiment of a system for providingposition-based haptic effects; and

FIG. 6 is a flow chart of steps for performing a method for providingposition-based haptic effects according to one embodiment.

DETAILED DESCRIPTION

Reference will now be made in detail to various and alternativeillustrative embodiments and to the accompanying drawings. Each exampleis provided by way of explanation and not as a limitation. It will beapparent to those skilled in the art that modifications and variationscan be made. For instance, features illustrated or described as part ofone embodiment may be used in another embodiment to yield a stillfurther embodiment. Thus, it is intended that this disclosure includemodifications and variations as come within the scope of the appendedclaims and their equivalents.

Illustrative Examples of Position-Based Haptic Effects

One illustrative embodiment of the present disclosure comprises a mobiledevice (e.g., a smart phone, tablet, e-reader, etc.). The mobile devicecomprises a touch-screen display configured to output a graphical userinterface (GUI) with one or more virtual buttons. The GUI may comprise,for example, a home screen that includes multiple icons associated withprograms stored on the mobile device. A user may interact with a virtualbutton, e.g., to execute a program or otherwise provide input to themobile device.

In the illustrative embodiment, the mobile device is configured toprovide substantially continuous haptic feedback to a user as the userapproaches and ultimately contacts the touch-screen display, e.g., tointeract with a virtual button. For example, in the illustrativeembodiment, the mobile device is configured to detect the distancebetween the user's finger and the touch-screen display. The mobiledevice is configured to substantially continuously output remote hapticeffects (e.g., puffs of air, an ultrasonic pressure wave, and/or laserbeam stimulation) to the user's finger as the user's finger moves closerand closer to the virtual button output on the touch-screen display. Insome embodiments, the mobile device outputs remote haptic effects thatprogressively increase in strength as the user approaches thetouch-screen display. This may simulate a spring force, or another formof resistance, typically associated with a real, physical button.Further, in the illustrative embodiment, the mobile device is configuredto output a local haptic effect (e.g., a click sensation) in response tothe user contacting the virtual button via the touch-screen display. Thelocal haptic effect may simulate the actuation of a physical button. Insuch an embodiment, providing continuous haptic feedback as the userapproaches and ultimately contacts the virtual button output on thetouch-screen display may more accurately simulate a real interactionwith a physical button, and/or provide a three-dimensional feel to anotherwise two-dimensional user interface.

In another illustrative embodiment, the mobile device comprises a GUIthat includes multiple user interface levels with which the user caninteract. The mobile device is configured to activate different userinterface levels as the user approaches and ultimately contacts thetouch-screen display. The user can interact with a particular userinterface level by performing a gesture at a distance (from the mobiledevice) associated with the user interface level.

For example, in some embodiments, the mobile device may execute amedical simulation comprising a GUI with multiple user interface levels.Each user interface level may be configured to output an image of,and/or provide information associated with, different facets of a bodypart (e.g., a human arm). For example, the mobile device may activate afirst user interface level in response to detecting the user's finger isat a far distance from the mobile device. The first user interface levelmay output an image of an exterior of the body part (e.g., skin, hair,etc.). The mobile device may activate a second user interface level inresponse to detecting the user's finger is at a medium distance from themobile device. The second user interface level may output an image oftissue and/or ligaments associated with the body part with key featureslabeled. The mobile device may activate a third user interface level inresponse to the user contacting the mobile device. The third userinterface level may output an image of bones associated with the bodypart. Thus, as the user approaches and/or contacts the mobile device,the mobile device can activate “deeper” user interface levels, throughwhich the user can access progressively more and/or differentinformation. This may additionally or alternatively provide athree-dimensional feel to an otherwise two-dimensional interface.

In some embodiments, the mobile device is configured to providesubstantially continuous haptic feedback associated with the userinterface levels as the user approaches and contacts the touch-screendisplay. For instance, in the above medical simulation example, as theuser approaches and contacts the touch-screen display, the mobile devicemay output substantially continuous haptic feedback configured tosimulate pushing a finger deeper through the different layers of a realbody part. In such an embodiment, the mobile device may output a hapticeffect configured to simulate skin and/or hair in response to detectingthe user's finger is at a distance associated with the first userinterface level. The mobile device may output a haptic effect configuredto simulate tissue and/or ligaments in response to detecting the user'sfinger is at a distance associated with the second user interface level.The mobile device may output a haptic effect configured to simulate bonein response to detecting that the user's finger contacted the mobiledevice. The combination of the haptic effects, e.g., as provided in asubstantially continuous manner, may more realistically simulate areal-world interaction with the body part.

The description of the illustrative embodiment above is provided merelyas an example. Various other embodiments of the present invention aredescribed herein and variations of such embodiments would be understoodby one of skill in the art. Advantages offered by various embodimentsmay be further understood by examining this specification and/or bypracticing one or more embodiments of the claimed subject matter.

Illustrative Systems for Position-Based Haptic Effects

FIG. 1A shows an embodiment of a system for providing position-basedhaptic effects. In the embodiment, the system comprises a computingdevice 100 configured to provide substantially continuous hapticfeedback as a user approaches and ultimately contacts the computingdevice 100. Additionally or alternatively, the computing device 100 maybe configured to provide substantially continuous haptic feedback ascomputing device 100 approaches and ultimately contacts the user. Asused herein, substantially continuous haptic feedback comprises one ormore haptic effects provided substantially throughout the duration of agesture performed by a user.

A gesture is any movement and/or positioning of the body and/or aphysical object that conveys meaning or user intent. It will berecognized that simple gestures may be combined to form more complexgestures. For example, bringing a finger into contact with a surface maybe referred to as a “finger on” gesture, while removing a finger fromthe surface may be referred to as a separate “finger off” gesture. Ifthe time between the “finger on” and “finger off” gestures is relativelyshort, the combined gesture may be referred to as “tapping”; if the timebetween the “finger on” and “finger off” gestures is relatively long,the combined gesture may be referred to as “long tapping”; if thedistance between the two dimensional (x, y) positions of the “finger on”and “finger off” gestures is relatively large, the combined gesture maybe referred to as “swiping”; if the distance between the two dimensional(x, y) positions of the “finger on” and “finger off” gestures isrelatively small, the combined gesture may be referred to as “smearing”,“smudging”, or “flicking” Gestures can additionally or alternatively bethree dimensional. For example, a gesture may comprise positioning abody part and/or a physical object in a particular location in realspace. In some embodiments, if the distance between three dimensional(x, y, z) positions during a finger movement (e.g., in real space) isrelatively large, the combined gesture may be referred to as “swiping.”If the distance between three dimensional (x, y, z) positions during afinger movement is relatively small, the combined gesture may bereferred to as “smearing”, “smudging”, or “flicking.” Any number of twodimensional or three dimensional simple or complex gestures may becombined in any manner to form any number of other gestures. A gesturecan also be any form movement or positioning (e.g., of a body part orphysical object) recognized by the computing device 100 and converted toelectronic signals. Such electronic signals can activate a hapticeffect, such as substantially continuous haptic feedback, where aposition sensor 104 captures the user intent that generates a hapticeffect.

In some embodiments, the computing device 100 is configured to providesubstantially continuous haptic feedback to guide the user to a locationof a particular virtual object (e.g., an enabled virtual button) outputon the touch-screen display 102. For example, in such embodiment, thecomputing device 100 is configured to output a remote haptic effect(e.g., via haptic output device 106) in response to detecting the userpositioned above the particular virtual object. The remote haptic effectmay be perceptible by the user when the user is not contacting thecomputing device 100. The remote haptic effect may help the userinitially locate the virtual object. In such an embodiment, thecomputing device 100 is further configured to continuously output remotehaptic effects to the user as the user gestures toward the touch-screendisplay 102 while remaining positioned overtop of the virtual object.This may notify the user that the user is approaching the virtualobject. In such an embodiment, the computing device 100 is alsoconfigured to output a local haptic effect in response to detecting theuser contacting the virtual button via the touch-screen display 102. Alocal haptic effect may be perceptible by the user when the user iscontacting the computing device 100. The local haptic effect may notifythe user that the user interacted with the virtual object. Thecontinuous haptic feedback as the user hovers over, approaches, andultimately contacts the virtual object may guide the user to the virtualobject, e.g., without the user having to look at the touch-screendisplay 102.

As a more specific example, in some embodiments, the computing device100 is configured to output a first haptic effect (e.g., via hapticoutput device 106) in response to detecting a gesture over the virtualobject at a first distance from the computing device 100 (e.g., withindistance range 112 from the computing device 100). In such anembodiment, the computing device 100 is configured to output a secondhaptic effect (e.g., via haptic output device 108) in response todetecting a gesture over the virtual object at a second distance fromthe computing device 100 (e.g., within distance range 114, but notcontacting computing device 100). Further still, in such an embodiment,the computing device 100 is configured to output a third haptic effect(e.g., via haptic output device 110) in response to detecting a gestureover the virtual object at a third distance from the computing device100 (e.g., contacting the computing device 100 and/or touch-screendisplay 102). The first haptic effect, second haptic effect, and/orthird haptic effect may be the same as or different from one another.For example, the first haptic effect may comprise an air puff, thesecond haptic effect may comprise a static ESF effect (e.g., asdescribed in greater detail with respect to FIG. 2), and the thirdhaptic effect may comprise a vibration.

In some embodiments, the computing device 100 configures thecharacteristics of (e.g., the waveform, type, amplitude, duration,frequency, and/or time of output) at least two haptic effects such thatthe transition between the at least two haptic effects is perceptiblyseamless to the user. For example, the computing device 100 mayconfigure the characteristics of the first haptic effect, second hapticeffect, and third haptic effect such that transitions between the hapticeffects are perceptibly seamless to the user. This may provide the userwith a substantially consistent haptic experience, e.g., throughout agesture. For example, this may provide the user with a substantiallyconsistent haptic experience as the user approaches, contacts, and/ormoves away from the computing device 100. In some embodiments, thecomputing device 100 configures the characteristics of at least twohaptic effects such that the at least two haptic effects are clearlydistinguishable to the user. For example, the computing device 100 mayconfigure the characteristics of the first haptic effect, second hapticeffect, and third haptic effect such that at least one transitionbetween the haptic effects is distinctly perceptible to the user. Thismay provide the user with information, such as a distance between theuser's body part and the computing device 100.

Embodiments may output any number and configuration of haptic effects toprovide substantially continuous haptic feedback throughout a gesture.For example, in some embodiments, the computing device 100 outputs afourth haptic effect in response to detecting another gesture withinanother distance range of the computing device 100, a fifth hapticeffect and a sixth haptic effect in response to detecting still anothergesture within still another distance range of the computing device 100,and so on.

Although the embodiments discussed above are with respect to a userapproaching and ultimately contacting the computing device 100, someembodiments may provide substantially continuous haptic feedbackthroughout any gesture that includes at least one position in which theuser is not contacting a surface (e.g., of the computing device 100) andone position in which the user is contacting the surface. For example,some embodiments may provide substantially continuous haptic feedbackthroughout a gesture in which the user approaches the computing device100 with a finger, contacts the computing device 100 with the finger,and then lifts the finger away from the computing device 100. Thus, thesubstantially continuous haptic feedback can start and end with the usernot contacting the computing device 100.

As another example, some embodiments provide substantially continuoushaptic feedback throughout a gesture that begins after the user contactsthe computing device 100 and ends after the user lifts the finger off ofthe computing device 100. For instance, the user may interact with asurface (e.g., a touch-screen display 102) of the computing device 100with a body part. The computing device 100 may detect the userinteraction and output a local haptic effect. The user may then lift thebody part off of the computing device 100 (e.g., when the user is doneinteracting with the computing device 100). In some embodiments, thecomputing device 100 may continue to output haptic feedback to the user(e.g., in the form of remote haptic effects), e.g., while the user'sbody part is within the distance range 114 and/or distance range 112from the computing device 100. The computing device 100 may output suchhaptic feedback until, e.g., a distance between the user's body part andthe computing device 100 exceeds a threshold. Thus, the substantiallycontinuous haptic feedback can start while the user is contacting thecomputing device 100 and end while the user is not contacting thecomputing device 100.

In some embodiments, the computing device 100 may use the same class ofhaptic effects (e.g., static ESF haptic effects) to output differentsensations to a user at different positions, e.g., throughout a gesture.For example, the computing device 100 may be configured to output afirst static ESF effect (e.g., configured to simulate a low-magnitudevibration) in response to detecting a gesture within distance range 112from the computing device 100, a second static ESF effect (e.g.,configured to simulate a medium-magnitude vibration) in response todetecting a gesture within distance range 114 from the computing device100, and/or a third static ESF effect (e.g., a high-magnitude vibration)in response to detecting a gesture including contacting the computingdevice 100.

FIG. 1B shows another embodiment of a system for providingposition-based haptic effects. In the embodiment, the system comprises acomputing device 100 configured to output a user interface (e.g., viatouch-screen display 102) comprising multiple user interface levels withwhich a user can interact. A user interface may comprise any number ofuser interface levels. Further, the computing device 100 may associateany number and/or configuration of user positions throughout a gesturewith any number and/or configuration of user interface levels. Forexample, in some embodiments, the mobile device may associate threedifferent user positions throughout a gesture with the same userinterface level. In some embodiments, the user interface levels can besimulated as overlaying one another, for example, to provide a simulateddepth to the user interface. In some embodiments, the computing device100 is configured to execute one or more functions associated with eachuser interface level.

In the embodiment shown in FIG. 1B, the computing device 100 isoutputting a virtual map associated with a navigation application. Asthe user approaches and/or contacts the computing device 100 (e.g., asshown by a dashed line), the computing device 100 is configured to cyclethrough one or more user interface levels, e.g., configured to providethe user with different levels of detail associated with the virtualmap. For example, in response to the user positioning a finger withindistance range 112 of the computing device 100, the computing device 100may activate a first user interface level configured to, e.g., outputinformation associated with states or cities on the virtual map. As theuser moves the finger to within the second distance range 114 of, butnot contacting, the computing device 100, the computing device 100 mayactivate a second user interface level configured to, e.g., add detailto the virtual map, such as town or street names. In response to theuser contacting the computing device 100, the computing device 100 mayactivate a third user interface level configured to, e.g., zoom in on aparticular location on the virtual map. Thus, the computing device 100may activate different user interface levels associated with the virtualmap in response to detecting the user gesturing within differentdistance ranges of the computing device 100.

In some embodiments, the computing device 100 is configured to determinea haptic effect to output based at least in part on the user interfacelevel and/or the gesture. Such haptic effects may be configured to,e.g., notify the user of the particular user interface level with whichthe user is interacting and/or that the user switched between userinterface levels. For example, the computing device 100 may output afirst haptic effect in response to activating the first user interfacelevel that is clearly distinguishable from a second haptic effectassociated with activating the second user interface level. Thedistinguishable haptic effects may notify the user that the user hasswitched between user interface levels.

In some embodiments, the computing device 100 is configured to outputhaptic effects configured to indicate a precision with which thecomputing device 100 detected a gesture. For example, the computingdevice 100 may detect a gesture (e.g., or a portion of a gesture, suchas a particular position of the user) with less precision when thegesture is performed is at a larger distance from the computing device100. For instance, when the gesture is performed at a distance of 1meter from the computing device 100, the computing device 100 may detecta position associated with the gesture to within 1 centimeter of theactual position. When the gesture is at a distance of 1 centimeter fromthe computing device 100, the computing device 100 may detect a positionassociated with the gesture to within one millimeter of the actualposition. In some embodiments, the computing device 100 outputs a hapticeffect that the user perceives as fuzzy in response to detecting thegesture with lower precision. The computing device 100 may output ahaptic effect that the user perceives as sharp in response to detectingthe gesture with higher precision. In some embodiments, the computingdevice 100 outputs a haptic effect to a larger surface area of theuser's body in response to detecting the gesture with lower precision.The computing device 100 can output a haptic effect to a smaller surfacearea of the user's body in response to detecting the gesture with higherprecision.

FIG. 2 is a block diagram showing a system for position-based hapticeffects according to one embodiment. The computing device 201 maycomprise a laptop computer, desktop computer, game controller, gamepad,remote control, medical device, car computing device (e.g., a computerfor controlling one or more automobile systems or devices such asstereo, HVAC, lighting, navigation, or other vehicle functions), wand,stylus, pen, and/or a portable gaming device. In some embodiments, thecomputing device 201 is associated with a wearable device (e.g., a ring,a shoe, an armband, a sleeve, a jacket, glasses, a glove, a watch, awristband, a bracelet, an article of clothing, a hat, a headband, and/orjewelry) and configured to be worn by a user and/or coupled to a user'sbody.

In some embodiments, the components (e.g., the processor 202, networkinterface device 210, haptic output device 218, sensors 230, positionsensor 232, touch sensitive surface 216, etc.) of the computing device201 may be integrated into a single housing. In other embodiments, thecomponents may be distributed (e.g., among multiple housings orlocations) and in electrical communication with one another. Thecomputing device 201 may or may not comprise all of the componentsdepicted in FIG. 2. For example, in some embodiments, the computingdevice 201 may not comprise the sensor 230.

The computing device 201 comprises a processor 202 interfaced with otherhardware via bus 206. A memory 204, which can comprise any suitabletangible (and non-transitory) computer-readable medium such as RAM, ROM,EEPROM, or the like, may embody program components that configureoperation of the computing device 201. In some embodiments, thecomputing device 201 may further comprise one or more network interfacedevices 210, input/output (I/O) interface components 212, and additionalstorage 214.

Network interface device 210 can represent one or more of any componentsthat facilitate a network connection or otherwise facilitatecommunication between electronic devices. Examples include, but are notlimited to, wired interfaces such as Ethernet, USB, IEEE 1394, and/orwireless interfaces such as IEEE 802.11, Bluetooth, near-fieldcommunication (NFC) interfaces, RFID interfaces, or radio interfaces foraccessing cellular telephone networks (e.g., transceiver/antenna foraccessing a CDMA, GSM, UMTS, or other mobile communications network).

I/O components 212 may be used to facilitate connection to devices suchas one or more displays, touch sensitive surfaces 216, keyboards, mice,speakers, microphones, buttons, and/or other hardware used to input dataor output data. Storage 214 represents nonvolatile storage such asread-only memory, flash memory, ferroelectric RAM (F-RAM), magnetic,optical, or other storage media included in the computing device 201 orcoupled to processor 202.

The computing device 201 may comprise a touch sensitive surface 216. Insome embodiments, the touch sensitive surface 216 is flexible ordeformable. Touch sensitive surface 216 represents any surface that isconfigured to sense tactile input of a user. One or more touch sensors208 are configured to detect a touch in a touch area (e.g., when anobject, such as a user's finger or a stylus, contacts a touch sensitivesurface 216) and transmit signals associated with the touch to processor202. Any suitable number, type, or arrangement of touch sensors 208 canbe used. For example, in some embodiments, resistive and/or capacitivesensors may be embedded in touch sensitive surface 216 and used todetermine the location of a touch and other information, such aspressure, speed, and/or direction of the touch. In some embodiments,capacitive sensors may detect the proximity of a user's finger to thetouch sensor 208 (e.g., embedded in the touch sensitive surface 216).For example, the touch sensor 208 may comprise a capacitive sensorconfigured to detect a change in capacitance as a user's fingerapproaches the touch sensor 208. The touch sensor 208 may determinewhether the user's finger is within a particular distance of the touchsensor 208 based on the change in capacitance.

The touch sensor 208 can additionally or alternatively comprise othertypes of sensors. For example, optical sensors with a view of the touchsensitive surface 216 may be used to determine the touch position. Asanother example, the touch sensor 208 may comprise a LED (Light EmittingDiode) finger detector mounted on the side of a display. In someembodiments, touch sensor 208 may be configured to detect multipleaspects of the user interaction. For example, touch sensor 208 maydetect the speed, pressure, and direction of a user interaction, andincorporate this information into the signal transmitted to theprocessor 202.

In some embodiments, the computing device 201 comprises a touch-enableddisplay that combines a touch sensitive surface 216 and a display of thedevice. The touch sensitive surface 216 may correspond to the displayexterior or one or more layers of material above components of thedisplay. In other embodiments, touch sensitive surface 216 may notcomprise (or otherwise correspond to) a display, depending on theparticular configuration of the computing device 201.

In some embodiments, the computing device 201 comprises one or moresensor(s) 230. The sensor(s) 230 are configured to transmit sensorsignals to the processor 202. The sensor(s) 230 may comprise, forexample, a range sensor, depth sensor, biosensor, camera, and/ortemperature sensor. The computing device 201 may determine one or morehaptic effects based at least in part on sensor signals from sensor 230.For example, in some embodiments, the computing device 201 may use datafrom sensor(s) 230 to determine one or more characteristics of a user'sbody part to which a haptic effect will be applied. The computing device201 may determine one or more characteristics of the haptic effect basedon the body part, e.g., to improve the quality of the perceived hapticeffect and/or prevent injury to the user.

For example, in some embodiments, the computing device 201 executes agame, such as a war game. The computing device 201 may be configured tooutput a haptic effect (e.g., a remote haptic effect) in response to agame event, such as a virtual explosion. In some embodiments, thecomputing device 201 may receive one or more images from sensor 230(e.g., a camera in view of the user's body). The computing device 201may analyze the images and determine that a destination location on auser's body for a haptic effect comprises, e.g., hair. In someembodiments, a user may perceive a haptic effect on a body partcomprising hair as weaker than on a body part without hair. In such anembodiment, the computing device 201 may determine and output a hapticeffect based on the detected hair. For example, the computing device 201may determine and output a haptic effect comprising an increasedamplitude. This may improve the quality of the haptic effect perceivedby the user.

As another example, in some embodiments, the computing device 201 maydetermine that a body part to which a haptic effect (e.g., aprojected/remote haptic effect) is to be applied comprises, e.g., aparticularly sensitive body part (e.g., an ear or eye). For example, thecomputing device 201 may analyze one or more images from sensor 230 anddetermine, based on the images, that the computing device 201 willoutput the haptic effect to a particular body part. The computing device201 may determine, e.g., using a lookup table and/or algorithm, that theparticular body part comprises a sensitive body part. For example, thecomputing device 201 may use a lookup table to map the particular bodypart to a corresponding sensitivity level. The computing device 201 mayresponsively determine a haptic effect with one or more characteristics(e.g., a type, amplitude, waveform, frequency, or other characteristic)configured to prevent and/or reduce the likelihood of injury to the bodypart.

For example, the computing device 201 may be executing a game simulatingvirtual bugs. In some embodiments, the computing device 201 maydetermine and output haptic effects configured to simulate bugs crawlingon the user, such as on the user's ear. For example, the computingdevice may determine a remote haptic effect comprising a puff of air (oran ultrasonic wave) that is to be emitted toward the user's ear, e.g.,to simulate a bug on the user's ear. In some embodiments, the computingdevice 201 may determine that the user's ear is a sensitive body part.The computing device 201 may responsively alter a characteristic of thehaptic effect to decrease the likelihood of injuring the user. Forexample, the computing device 201 may decrease the amplitude orintensity of the puff of air (or the ultrasonic wave), e.g., to reducethe likelihood of injuring the user's hearing.

As another example, the computing device 201 may be executing a militarygame in which a user can control a virtual character. In response to thevirtual character getting shot, the computing device 201 may beconfigured to output a haptic effect (e.g., a remote haptic effect) to aportion of the user's body associated with where the virtual charactergot shot. For example, the computing device 201 may be configured toproject a hard material (e.g., a plastic pellet) toward the portion ofthe user's body associated with where the virtual character got shot.This may simulate the gun shot. In some embodiments, the computingdevice 201 may determine that the haptic effect is to be applied to, orprojected toward, a sensitive portion of the user's body, such as theuser's eye. In some embodiments, the computing device 201 mayresponsively change a haptic effect from one type (e.g., a remote hapticeffect comprising an emission of a hard material, such as a plasticpellet) to another type (e.g., a remote haptic effect comprising anemission of a gas, such as a jet of air). This may reduce the likelihoodof injury to the user's eye.

In some embodiments, the computing device 201 may determine that thereis a risk that the haptic effect will be applied to, or projectedtoward, a sensitive body part. For example, in the above military gameembodiment, the computing device 201 may determine a remote hapticeffect configured to be projected toward the user's nose. For instance,the computing device 201 may determine that a plastic pellet is to beprojected toward the user's nose in response to the user's virtualcharacter getting shot in the nose. But the computing device 201 mayalso determine that the user is moving, increasing the likelihood thatthe projected material may contact a sensitive body part, such as theuser's eye. The computing device 201 may responsively change the hapticeffect from one type (e.g., a remote haptic effect comprising anemission of a hard material, such as a plastic pellet) to another type(e.g., a remote haptic effect comprising an emission of a soft material,such as foam). This may decrease the likelihood of injury to thesensitive body part.

In some embodiments, the sensor 230 (e.g., a temperature sensor) may beconfigured to detect a temperature of the user (e.g., a skin temperatureof the user). In some embodiments, the computing device 201 may modify acharacteristic of a haptic effect based on the temperature. For example,the computing device 201 may be configured to output a haptic effectcomprising applying heat to at least a portion the user's body (e.g.,the user's stomach) in response to, e.g., a game event, such as avirtual explosion. In such an embodiment, the computing device 201 maydetermine no haptic effect in response to a sensor signal from sensor230 indicating, e.g., that the user's body temperature is above athreshold. For example, the computing device 201 may not output thehaptic effect in response to determining that the user's body is toohot. This may prevent and/or reduce the likelihood of injury (e.g., heatstroke) to the user.

The computing device 201 is in communication with a position sensor 232.In some embodiments, the position sensor 232 comprises a camera, a 3Dimaging system (e.g., the 3D imaging system commonly sold under thetrademark Microsoft Kinect®), a depth sensor, an ultrasonic transducer,an accelerometer, a gyroscope, a radio frequency identification (RFID)tag or reader, an indoor proximity system, a NFC communication device, aglobal positioning system (GPS) device, a magnetometer, a wirelessinterface (e.g., an IEEE 802.11 or Bluetooth interface), an infraredsensor, a range sensor, and/or a LED-based tracking system. Positionsensor 232 is configured to detect one or more positions of a user'sbody part and/or a physical object and transmit one or more associatedsensor signals to the processor 202. The position can be athree-dimensional position in real space or a relative position withrespect to the computing device 201. In some embodiments, the positionsensor 232 is configured to detect a three-dimensional gesture in realspace, e.g., based on a plurality of detected three-dimensionalpositions of a user's body part and/or a physical object.

In some embodiments, the processor 202 is configured to determine agesture based on one or more sensor signals from the position sensor232. For example, the user may make a gesture (e.g., a swiping gesturein real space). The computing device 201 may analyze a plurality ofcamera images associated with the gesture (e.g., taken by the positionsensor 232) and determine one or more characteristics of the gesture(e.g., the type of gesture) based on the plurality of camera images. Forexample, the processor 202 may determine that the user made a particulartype of gesture based on the camera images. As another example, in someembodiments, the position sensor 232 comprises a wireless interface thatis configured to detect the strength of a wireless signal emitted by anelectronic device worn by the user and/or coupled to the user's body(e.g., held by the user). The position sensor 232 may transmit a sensorsignal associated with the wireless signal strength to the processor202. Based on a plurality of different wireless signal strengthsdetected by the position sensor 232 over a period of time, the processor202 may determine, for example, that the electronic device moved in aparticular pattern associated with a particular gesture.

In some embodiments, the position sensor 232 is external to computingdevice 201 and in wired or wireless communication with the computingdevice 201 (e.g., as shown with respect to FIG. 3). For example, theposition sensor 232 may comprise a camera associated with a wearabledevice (e.g., glasses or a tie) and in communication with the computingdevice 201. As another example, the position sensor 232 may comprise a3D imaging system and/or a LED-based tracking system positioned externalto the computing device 201 (e.g., on a shelf in a user's home) and incommunication with the computing device 201.

In some embodiments, the computing device 201 comprises a haptic outputdevice 218 in communication with processor 202. In some embodiments, asingle haptic output device 218 is configured to output both local andremote haptic effects. The haptic output device 218 is configured tooutput a haptic effect (e.g., a remote haptic effect and/or a localhaptic effect) in response to a haptic signal. A haptic effect maycomprise, for example, a change in temperature, a stroking sensation, anelectro-tactile effect, a change in temperature, and/or a surfacedeformation (e.g., a deformation of a surface associated with thecomputing device 201). Further, some haptic effects may use multiplehaptic output devices 218 of the same or different types in sequenceand/or in concert. Although a single haptic output device 218 is shownin FIG. 2, embodiments may use multiple haptic output devices 218 of thesame or different type to produce haptic effects.

In some embodiments, the haptic output device 218 is external tocomputing device 201 and in communication with the computing device 201(e.g., via wired interfaces such as Ethernet, USB, IEEE 1394, and/orwireless interfaces such as IEEE 802.11, Bluetooth, or radiointerfaces). For example, the haptic output device 218 may be associatedwith (e.g., coupled to) a wearable device and configured to receivehaptic signals from the processor 202.

In some embodiments, the haptic output device 218 is configured tooutput a haptic effect comprising a vibration. The haptic output device218 may comprise, for example, one or more of a piezoelectric actuator,an electric motor, an electro-magnetic actuator, a voice coil, a shapememory alloy, an electro-active polymer, a solenoid, an eccentricrotating mass motor (ERM), a linear resonant actuator (LRA), and/or aplurality of conductive electrodes configured to vibrate due toelectrostatic attraction and repulsion between the electrodes.

In some embodiments, the haptic output device 218 is configured tooutput a haptic effect modulating the perceived coefficient of frictionof a surface associated with the haptic output device 218. In oneembodiment, the haptic output device 218 comprises an ultrasonicactuator. An ultrasonic actuator may vibrate at an ultrasonic frequency,for example 20 kHz, increasing or reducing the perceived coefficient ofan associated surface. In some embodiments, the ultrasonic actuator maycomprise a piezo-electric material.

In some embodiments, the haptic output device 218 uses electrostaticattraction to output an electrostatic haptic effect. In someembodiments, the electrostatic haptic effect may comprise a simulatedtexture, a simulated vibration, a stroking sensation, and/or a perceivedchange in a coefficient of friction on a surface associated withcomputing device 201.

For example, in some embodiments, the haptic output device 218 maycomprise a conducting layer and an insulating layer. In someembodiments, the conducting layer may comprise any semiconductor orother conductive material, such as copper, aluminum, gold, silver,graphene, carbon nanotubes, Indium tin oxide (ITO), platinum,poly(3,4-ethylenedioxythiophene) polystyrene sulfonate (“PEDOT:PSS), orany combination of these. In some embodiments, the insulating layer maycomprise glass, plastic, polymer, silica, parylene, kapton tape, silicondioxide (SiO2), Aluminum Oxide (Al2O3), silicon nitride (Si3N4), or anycombination of these. Furthermore, the processor 202 may operate thehaptic output device 218 by applying an electric signal, for example anAC signal, to the conducting layer. In some embodiments, a high-voltageamplifier may generate the AC signal. The electric signal may generate acapacitive coupling between the conducting layer and an object (e.g., auser's finger, head, foot, arm, shoulder, leg, or other body part, or astylus) near or touching the haptic output device 218. Varying thelevels of attraction between the object and the conducting layer canvary the haptic effect perceived by a user.

In some embodiments, the conducting layer comprises a plurality ofelectrodes. The plurality of electrodes may be arranged in a particularpattern or configuration. For example, the plurality of electrodes maycomprise a plurality of electrode strips. The electrode strips may bearranged in a diagonal pattern, a horizontal pattern, a verticalpattern, or another pattern across one or more surfaces of the computingdevice 201. As another example, one or more of the electrodes maycomprise a circular, triangular, oval, square, rectangular, or othershape. For example, a perimeter or circumference of the electrodes maycomprise a circular, triangular, oval, square, rectangular, or othershape. The plurality of electrodes may comprise any number ofelectrodes, arranged in any configuration, and including any number ofshapes. In some embodiments, the processor 202 can actuate the hapticoutput device 218 by applying an electrical signal to all, or a subsetof, the plurality of electrodes. The electric signal may generate acapacitive coupling between the electrodes and an object near orcontacting the electrodes. A user may perceive the capacitive couplingas a haptic effect.

In some embodiments, the electrostatic haptic effect comprises a“dynamic ESF effect.” A dynamic ESF effect may comprise an electrostatichaptic effect perceptible to a user upon a user's body part (e.g., afinger) moving relative to a surface associated with the haptic outputdevice 218. For example, a dynamic ESF effect may be perceptible to theuser upon the user sliding a finger along the surface of the insulatorlayer of the haptic output device 218. As another example, a dynamic ESFeffect may be perceptible to the user upon the computing device 201moving against the user's body (e.g., while the user remains still).

For example, the computing device 201 may output a graphical userinterface (GUI) on a touch-screen display comprising one or more virtualuser interface components (e.g., buttons, sliders, knobs, etc.). Forinstance, the GUI may comprise a virtual button. The user interfacecomponent may comprise a virtual texture, such as a plastic texture. Insome embodiments, the computing device 201 may output a haptic effect inresponse to the user sliding a finger across a location on thetouch-screen display associated with the user interface component. Thehaptic effect may comprise a dynamic ESF effect configured to, e.g.,simulate the plastic texture.

In some embodiments, the electrostatic haptic effect comprises a staticESF effect. A static ESF effect may comprise an electrostatic hapticeffect perceptible to a user without the user having to move a body partwith respect to (e.g., across or perpendicular to) a surface associatedwith the haptic output device 218. For example, the user may besubstantially stationary with the surface and still perceive the hapticeffect. Further, the user may not need to contact the surface at all toperceive the static ESF effect. For example, in the above GUIembodiment, the computing device 201 may output a static ESF hapticeffect in response to the user hovering a finger above the touch-screendisplay and over the user interface component. In some embodiments, thestatic ESF effect comprises repeatedly attracting and repelling theuser's body part (e.g., finger) using electrostatic forces to, e.g.,generate a vibration sensation. The haptic effect may, for example,notify to the user that the user is approaching and/or hovering over anenabled button or a disabled button.

In some embodiments, the haptic output device 218 comprises adeformation device configured to output a deformation haptic effect. Thedeformation haptic effect may comprise raising or lowering portions of asurface associated with the computing device 201. For example, thedeformation haptic effect may comprise raising portions of a surface thecomputing device 201 to generate a bumpy texture. In some embodiments,the deformation haptic effect may comprise bending, folding, rolling,twisting, squeezing, flexing, changing the shape of, or otherwisedeforming a surface associated with the computing device 201. Forexample, the deformation haptic effect may apply a force on thecomputing device 201 or a surface associated with the computing device201, causing it to bend, fold, roll, twist, squeeze, flex, change shape,or otherwise deform. For example, in response to the user approachingthe computing device 201, the computing device 201 may output a hapticeffect configured to cause the computing device 201 to bend toward theuser. This deformation haptic effect may be at least a part ofsubstantially continuous haptic feedback provided to the user as theuser approaches and ultimately contacts the computing device 201.

In some embodiments, the haptic output device 218 comprises fluid orother materials configured for outputting a deformation haptic effect(e.g., for bending or deforming the computing device 201 or a surfaceassociated with the computing device 201). For example, the fluid maycomprise a smart gel. A smart gel comprises a material with mechanicalor structural properties that change in response to a stimulus orstimuli (e.g., an electric field, a magnetic field, temperature,ultraviolet light, shaking, or a pH variation). For instance, inresponse to a stimulus, a smart gel may change in stiffness, volume,transparency, and/or color. In some embodiments, stiffness may comprisethe resistance of a surface associated with the computing device 201against deformation. In some embodiments, one or more wires may beembedded in or coupled to the smart gel. As current runs through thewires, heat is emitted, causing the smart gel to expand or contract.This may cause the computing device 201 or a surface associated with thecomputing device 201 to deform.

As another example, the fluid may comprise a rheological (e.g., amagneto-rheological or electro-rheological) fluid. A rheological fluidcomprises metal particles (e.g., iron particles) suspended in a fluid(e.g., oil or water). In response to an electric or magnetic field, theorder of the molecules in the fluid may realign, changing the overalldamping and/or viscosity of the fluid. This may cause the computingdevice 201 or a surface associated with the computing device 201 todeform.

In some embodiments, the haptic output device 218 comprises a mechanicaldeformation device. For example, in some embodiments, the haptic outputdevice 218 may comprise an actuator coupled to an arm that rotates adeformation component. The deformation component may comprise, forexample, an oval, starburst, or corrugated shape. The deformationcomponent may be configured to move a surface associated with thecomputing device 201 at some rotation angles but not others. Theactuator may comprise a piezo-electric actuator, rotating/linearactuator, solenoid, an electroactive polymer actuator, macro fibercomposite (MFC) actuator, shape memory alloy (SMA) actuator, and/orother actuator. As the actuator rotates the deformation component, thedeformation component may move the surface, causing it to deform. Insuch an embodiment, the deformation component may begin in a position inwhich the surface is flat. In response to receiving a signal fromprocessor 202, the actuator may rotate the deformation component.Rotating the deformation component may cause one or more portions of thesurface to raise or lower. The deformation component may, in someembodiments, remain in this rotated state until the processor 202signals the actuator to rotate the deformation component back to itsoriginal position.

Further, other techniques or methods can be used to deform a surfaceassociated with the computing device 201 and/or otherwise generatehaptic effects. For example, the haptic output device 218 may comprise aflexible surface layer configured to deform its surface or vary itstexture based upon contact from a surface reconfigurable hapticsubstrate (including, but not limited to, e.g., fibers and nanotubes).In some embodiments, the haptic output device 218 is deformed orotherwise generates haptic effects using e.g., a motor coupled to wires,air or pockets, resonant mechanical elements, micro-electromechanicalsystems (“MEMS”) elements or pumps, thermal fluid pockets, variableporosity membranes, or laminar flow modulation.

In some embodiments, the haptic output device 218 is configured toremotely project haptic effects to a user (e.g., to generate remotehaptic effects). For example, the haptic output device 218 may compriseone or more jets configured to emit materials (e.g., solids, liquids,gasses, or plasmas) toward the user (e.g., a body part of the user). Inone such embodiment, the haptic output device 218 comprises a gas jetconfigured to emit puffs or streams of oxygen, nitrogen, or another gaswith varying characteristics upon receipt of the haptic signal. Asanother example, the haptic output device 218 may comprise one or moreultrasonic transducers or speakers configured to project pressure wavesin the direction of the user. In one such embodiment, the processor 202may cause the haptic output device 218 to emit a concentrated pressurewave toward the user in response to the user gesturing in real space.The concentrated pressure wave may vibrate a portion of the user's body(e.g., the user's finger). The user may perceive the vibration as ahaptic effect.

In some embodiments, the haptic output device 218 may be a portion ofthe housing of the computing device 201. In other embodiments, thehaptic output device 218 may be housed inside a flexible housingoverlaying a surface associated with the computing device 201 (e.g., thefront or back of the computing device 201). In some embodiments, thehaptic output device 218 may itself be flexible.

Turning to memory 204, modules 224, 226, and 228 are depicted to showhow a device can be configured in some embodiments to provideposition-based haptic effects. In some embodiments, the gesturedetection module 224 comprises one or more algorithms or lookup tablesuseable by the processor 202 to determine, for example, a gesture by theuser.

In some embodiments, the gesture detection module 224 comprises code fordetecting a contact with the computing device 201. For example, in suchan embodiment, the gesture detection module 224 may sample the touchsensor 208 to determine if the user has contacted the touch sensitivesurface 216. In another such embodiment, the gesture detection module224 may analyze a plurality of pictures from position sensor 232 anddetermine if pixels corresponding to a body part of the user overlaypixels associated with the computing device 201. If so, the gesturedetection module 224 may determine, e.g., the user is contacting thecomputing device 201.

In some embodiments, the gesture detection module 224 comprises code foranalyzing data received via the network interface device 210 todetermine a position of the user with respect to the computing device201. For example, in some embodiments, the user may be wearing a device,such as a smart ring or smart watch, comprising a GPS sensor. In such anembodiment, the device may be configured to transmit signals associatedwith the GPS position of the device (e.g., the GPS position of a bodypart of the user) to the computing device 201. In some embodiments, thegesture detection module 224 comprises code for receiving the GPSposition from the device and comparing the GPS position to a GPSposition of the computing device 201 (e.g., as detected by sensor 230).This may allow the computing device 201 to determine a position of thedevice (e.g., the user) with respect to the computing device 201. Thegesture detection module 224 may comprise code for determining a gesturebased on the position.

Haptic effect determination module 226 represents a program componentthat analyzes data to determine a haptic effect to generate. The hapticeffect determination module 226 may comprise code that selects one ormore haptic effects to output using one or more algorithms or lookuptables. In some embodiments, the haptic effect determination module 226comprises code for determining one or more haptic effects to outputbased on a detected gesture. For example, the processor 202 may access alookup table stored in haptic effect determination module 226 to map aspecific gesture to a particular haptic effect, such as a puff of air oran electrostatic haptic effect.

In some embodiments, the haptic effect determination module 226comprises code that determines a plurality of haptic effects to outputusing one or more haptic output devices 218, e.g., to providesubstantially continuous haptic feedback throughout a gesture. Forexample, the haptic effect determination module 226 may comprise codefor selecting two or more different haptic output devices from among aplurality of available haptic output devices to use in combination withone another to provide the substantially continuous haptic feedback as auser approaches and contacts computing device 201.

Haptic effect generation module 228 represents programming that causesprocessor 202 to generate and transmit haptic signals to the hapticoutput device 218 to generate a selected haptic effect. For example, thehaptic effect generation module 228 may access stored waveforms orcommands to send to the haptic output device 218 to create the desiredhaptic effect. In some embodiments, the haptic effect generation module228 may comprise algorithms to determine the haptic signal. Further, insome embodiments, haptic effect generation module 228 may comprisealgorithms to determine target coordinates for the haptic effect (e.g.,coordinates for a location on a user's body to which to output a hapticeffect).

Although the modules 224, 226, 228 are depicted in FIG. 2 as programcomponents within the memory 204, in some embodiments, the modules 224,226, 228 may comprise hardware. For example, modules 224, 226, 228 maycomprise analog to digital converters, processors, microcontrollers,comparators, amplifiers, transistors, and other analog or digitalcircuitry.

FIG. 3 shows an embodiment of a system 300 for providing position-basedhaptic effects. The system 300 comprises a wearable computing device302, a graspable computing device 304, a position sensor 316, and/or aremote haptic output device 318 connected to a network 314. The wearablecomputing device 302 and/or graspable computing device may be configuredsubstantially the same as the computing device 201 of FIG. 2.

The computing devices 302, 304, position sensor 316, and/or remotehaptic output device 318 may directly communicate with each other and/ormay communicate with each other via the network 314. For example, thewearable computing device 302 may directly wirelessly communicate withthe graspable computing device 304 (e.g., using Bluetooth).

The system 300 can use any number, combination, and/or configuration ofwearable computing devices 302, graspable computing devices 304,position sensors 316, and/or remote haptic output devices 318 to providesubstantially continuous haptic feedback throughout a gesture. Forexample, in some embodiments, the wearable computing device 302,graspable computing device 304, the position sensor 316, and the remotehaptic output device 318 work together to provide substantiallycontinuous haptic feedback to the user, e.g., as the user approaches,contacts, and/or lifts a finger off of a user interface component of thegraspable computing device 304, such as a trigger 306, joystick 308,button 310, and/or directional pad 312.

As a particular example, in some embodiments, the user may be wearing avirtual reality headset to play a video game in which the user's visionis substantially blocked. In such an embodiment, the system 300 may beconfigured to output substantially continuous haptic feedback as theuser approaches and ultimately picks up the graspable computing device304, e.g., to guide the user to the graspable computing device 304,which the user may not otherwise be able to see. For example, theposition sensor 316 may detect the user approaching the graspablecomputing device 304 and transmit associated sensor signals to thewearable computing device 302 and/or the remote haptic output device318. The wearable computing device 302 and/or the remote haptic outputdevice 318 may responsively output haptic effects, e.g., with decreasingmagnitudes as the user gets closer to the graspable computing device304. In response to the user finally contacting the graspable computingdevice 304, the graspable computing device 304 may to output a hapticeffect, such as a vibration. In some embodiments, the combination ofhaptic effects from various components of the system 300 may providesubstantially continuous haptic feedback to the user, e.g., to guide theuser toward the graspable computing device 304.

In some embodiments, the system 300 is configured to output one or morehaptic effects configured to provide information to the user. Forexample, the user may hold the graspable computing device 304, e.g., toplay a video game. The system 300 may detect a user's body part hoveringover, but not contacting, a particular user interface component of thegraspable computing device 304 (e.g., trigger 306). In some embodiments,the system 300 may responsively output a haptic effect (e.g., a puff ofair) toward the user (e.g., toward the back of the user's finger overthe trigger 306). The haptic effect may be configured to notify the userof, for example, a characteristic of the user interface component. Forinstance, in such an embodiment, the haptic effect may notify the userof the type, location, functionality, and/or status of the userinterface component, such as whether the user interface component isenabled. This may allow the user to determine, for example, whether aparticular user interface is enabled without physically contacting theuser interface component.

Further, in some embodiments, the system 300 is configured to providecontinuous haptic feedback as the user hovers over, approaches, andultimately contacts the user interface component. In some embodiments,the system 300 is configured to additionally or alternatively providecontinuous haptic feedback as the user moves a finger away from the userinterface component (e.g., after interacting with the user interfacecomponent). The haptic feedback output as the user moves away from theuser interface component may be different from the haptic feedbackoutput as the user approaches and/or contacts the user interfacecomponent. For example, system 300 may output haptic feedback as theuser moves away from the user interface component that is configured toindicate to the user that the user interface component is no longerenabled.

In some embodiments, the system 300 additionally or alternativelyoutputs a user interface (e.g., via a display) with multiple userinterface levels. For example, the system 300 may execute a video game(e.g., the graspable computing device 304 itself may execute the videogame) comprising a user interface. The video game may comprise a virtualobject, such as a virtual laser gun. In some embodiments, the system 300may detect the user interacting with and/or activating a first userinterface level of the user interface in response to a first gesture(e.g., the user positioning a body part and/or a physical object in afirst position with respect to the graspable computing device 304). Forexample, the system 300 may detect the user interacting with and/oractivating the first user interface level in response to the userhovering a finger over the trigger 306 of the graspable computing device304. In some embodiments, the system 300 may perform a function (e.g.,loading an ammunition magazine into the virtual laser gun) associatedwith the first user interface level. Additionally or alternatively, thesystem 300 may output a haptic effect associated with the first userinterface level, such as a haptic effect configured to simulate loadingan ammunition magazine into the virtual laser gun.

Further, in some embodiments, the system 300 may detect the userinteracting with and/or activating a second user interface level inresponse to the user positioning the body part in a second position withrespect to the graspable computing device 304. For example, system 300may detect the user may interacting with and/or activating the seconduser interface level in response to the user hovering the finger overthe trigger 306 more closely to the graspable computing device 304. Insome embodiments, the system 300 may perform a function (e.g., chargingup the virtual laser gun) associated with the second user interfacelevel. The system 300 may additionally or alternatively output a hapticeffect associated with the second user interface level, such as a hapticeffect configured to simulate charging the virtual laser gun.

Further still, in some embodiments, the system 300 may detect the userinteracting with and/or activating a third user interface level inresponse to the user positioning the body part in a third position withrespect to the graspable computing device 304. For example, the system300 may detect the user interacting with and/or activating the thirduser interface level in response to the user pressing the trigger 306 ofthe graspable computing device 304. In some embodiments, the system 300may perform a function (e.g., firing the virtual laser gun) associatedwith the third user interface level. The system 300 may additionally oralternatively output a haptic effect associated with the third userinterface level, such as a haptic effect configured to simulate firingthe virtual laser gun. Thus, the user may be able to perform a pluralityof functions by positioning a body part at different locations over aparticular user interface component. This may provide a more intuitiveand simplified experience for the user.

FIG. 4 shows another embodiment of a system 400 for providingposition-based haptic effects. The system 400 comprises a vehicle. Thevehicle comprises a computing device (e.g., internal to the vehicle).Although the vehicle in FIG. 4 comprises a car, in other embodiments,the vehicle may comprise a truck, boat, airplane, motorcycle, etc.

The computing device is in communication with one or more user interfacecomponents (e.g., joysticks, buttons, triggers, switches, knobs, touchsensitive surfaces, etc.). In the example shown in FIG. 4, the computingdevice is in communication with a touchscreen display 402 and multiplebuttons 406. The system 400 also comprises a position sensor 404 and ahaptic output device. The haptic output device is configured to providea user with local and/or remote haptic effects.

In some embodiments, the computing device is configured to provide theuser with haptic feedback (e.g., substantially continuous hapticfeedback) in response to a gesture in real space. For example, thecomputing device may output a GUI via the touchscreen display 402configured for, e.g., changing one or more vehicle settings, such as anaudio volume, radio station, air conditioning or heat level, and/or GPSnavigation setting. In some embodiments, the computing device isconfigured to provide the user with substantially continuous hapticfeedback as the user approaches, contacts, and/or moves away from aparticular virtual object of the GUI (e.g., a virtual button foradjusting an audio volume). In such an embodiment, the computing devicemay stop outputting the haptic feedback in response to detecting thatthe user is no longer gesturing over (e.g., hovering a finger over) thevirtual object, and/or that a distance between the virtual object andthe user exceeds a threshold (e.g., the user has moved far enough awayfrom the virtual object). In some embodiments, such continuous hapticfeedback may allow the user to locate a desired user interface componentand/or safely interact with the computing device, e.g., without visuallyfocusing on the touchscreen display, which may prevent an accident orinjury.

As an example, in some embodiments, the computing device is configuredto detect the user gesturing (e.g., with a finger) over a virtual buttonoutput on the GUI, e.g., for changing a radio station. The computingdevice is configured to responsively output a remote haptic effect(e.g., an ultrasonic pressure wave) configured to, e.g., notify the userthat the user is gesturing over the particular virtual object. In someembodiments, the computing device is configured to continue outputtingremote haptic effects in response to detecting the user approaching thevirtual button. In such an embodiment, the computing device may modulatethe strength of the remote haptic effects, e.g., such that the userperceives the remote haptic effects as having a substantially constantmagnitude. Thus, the user may perceive the remote haptic effects ashaving a substantially constant magnitude, e.g., even though the usermay be moving closer to the source of the remote haptic effects. In someembodiments, the computing device is configured to detect the usercontacting the virtual object (e.g., via the touchscreen display 402and/or position sensor 404). The computing device may responsively stopoutputting the remote haptic effects and/or output a local haptic effectcomprising, e.g., a pulsed vibration. This may confirm to the user thatthe computing device received the user input. In some embodiments, thecomputing device is configured to continuously output the local hapticeffect, e.g., until the computing device detects the user moving thebody part away from, and/or no longer contacting, the virtual object.The computing device may then output one or more remote haptic effects,e.g., until a distance between the user and the virtual object exceeds athreshold.

Although the above discussion is with reference to a virtual object(e.g., output on a touchscreen display 402), in some embodiments, thecomputing device may be configured to output haptic effects based ongestures performed with respect to one or more physical user interfacecomponents, such as buttons 406, triggers, switches, joysticks 408,knobs, etc.

In some embodiments, the computing device outputs a user interfacecomprising multiple user interface levels, e.g., via the touchscreendisplay 402. For example, in some embodiments, the computing device isconfigured to output a GUI associated with a radio in the car. In someembodiments, the user may interact with a first user interface level ofthe user interface by hovering a finger over a virtual button, such as aradio station scan button, output on the touchscreen display 402. Insome embodiments, the computing device may perform a function associatedwith the first user interface level. For example, the computing devicemay output (e.g., via the touchscreen display 402) a description of afunction associated with virtual button, such as “HOVER CLOSER TO SCAN,TAP TO TURN OFF.” The computing device may output a haptic effectconfigured to simulate, e.g., the presence or actuation of a physicalbutton. Further, in some embodiments, the user may interact with asecond user interface level by gesturing closer to the virtual button.In some embodiments, the computing device is configured to responsivelyscan through a plurality of radio stations. Further still, in someembodiments, the user may interact with a third user interface level bycontacting the touchscreen display 402 (e.g., and/or the virtual buttonoutput thereon). In some embodiments, the computing device is configuredto responsively turn off the radio. Thus, in some embodiments, computingdevice is configured to perform a number of functions in response to theuser interacting with various user interface levels.

FIG. 5 shows still another embodiment of a system for providingposition-based haptic effects. The system comprises a computing device502 (e.g., a smart phone or tablet) outputting a virtual object 506 on adisplay (e.g., a touchscreen display). The computing device 502 isoutputting (e.g., via a display) a virtual object 506 comprising, e.g.,fire. The computing device 502 is configured to detect a gesture, suchas the user approaching and/or moving away from the virtual object 506(e.g., as shown by the dashed arrow), and responsively output hapticeffects. The haptic effects are configured to provide a more realisticand/or immersive haptic experience for the user. For example, the hapticeffects may be configured to simulate, e.g., the heat of fire. Forinstance, in the example shown in FIG. 5, the computing device 502 isconfigured to output haptic effects (e.g., vibrations or projected heat)comprising increasing intensities in response to detecting the userapproaching the computing device 502 and/or decreasing intensities inresponse to detecting the user moving away from the computing device502. Further, in some embodiments, the computing device 502 isconfigured to output a local haptic effect, such as a jolt or shocksensation, in response to detecting the user contacting the computingdevice 502. In some embodiments, the combination of substantiallycontinuous remote and/or local haptic effects may more realisticallysimulate a characteristic of the virtual object, such as the increasingor decreasing heat of a fire as the user moves with respect to the fire.

As another example, in some embodiments, the computing device 502 isconfigured to output (e.g., via a display) a virtual object 506comprising, e.g., a magnet and/or force field. In such an embodiment,the computing device 502 may output haptic effects comprising increasingintensities in response to detecting the user approaching the computingdevice 502. For example, the haptic effect may comprise a stream offluid or gas configured to resist the movement of the user. This mayresist against the user approaching the virtual object 506, e.g.,simulating magnetic resistance or a force field. The computing device502 may output haptic effects comprising decreasing intensities inresponse to detecting the user moving away from the computing device502. For example, the haptic effect may comprise a stream of fluid orgas configured to aid the movement of the user away from the computingdevice 502. This may more realistically simulate the magnetic field ofthe magnet and/or the effects of the force field.

In some embodiments, the haptic effects are configured to provideinformation to the user. For example, the computing device 502 mayexecute an application. The application may execute a function which,e.g., may cause damage to the computing device 502 if the function isinterrupted. In some embodiments, the computing device 502 is configuredto detect the user gesturing toward the computing device 502 (e.g., viaposition sensor 504) and responsively output a remote haptic effectconfigured to, e.g., resist the user's movement toward the computingdevice 502. In some embodiments, the computing device 502 may increasethe intensity and/or amount of the resistance as the user continues toapproach the computing device 502. This may indicate to the user thatthe user should not interact with the computing device 502. In someembodiments, the computing device 502 is configured to detect the userultimately contacting the computing device 502 and responsively output ahaptic effect comprising, e.g., a series of pulsed vibrations. This mayindicate to the user that the user should not interact with thecomputing device 502. Further, in some embodiments, the computing device502 is configured to detect the user sliding a finger across the surfaceof the computing device 502 (e.g., toward a virtual button output on atouch-screen display) and responsively output a haptic effectcomprising, e.g., a perceived increase in a coefficient of friction.This may make it physically more difficult for the user to interact withthe computing device 502. The combination of haptic effects may moreadequately notify the user of a status, characteristic, and/or otherfeature of the application.

In some embodiments, the computing device 502 additionally oralternatively outputs a user interface comprising multiple userinterface levels, e.g., configured to simulate interacting with thevirtual object 506 at different depths and/or interacting with differentsurfaces of the virtual object 506. For instance, the virtual object 506may comprise a virtual fruit, such as a peach. The user may interactwith a portion of the virtual fruit, such as an outer layer, bygesturing (e.g., positioning a body part) in a first location in realspace. In some embodiments, the computing device 502 may detect thegesture and responsively, e.g., output a haptic effect configured tosimulate the fuzz or outer skin of the fruit. Further, user may interactwith another portion of the virtual fruit, such as an inner layer, bygesturing in a second location in real space. The second position may becloser to the computing device 502 than the first position. In someembodiments, the computing device 502 may detect the gesture in thesecond location and responsively, e.g., output a haptic effectconfigured to simulate an internal characteristic of the fruit, such asa squishy or soft texture. In this manner, the computing device 502 cansimulate different surfaces and/or depths of a virtual object 506 inresponse to different gestures.

Illustrative Methods for Providing Position-Based Haptic Effects

FIG. 6 is a flow chart of steps for performing a method for providingposition-based haptic effects according to one embodiment. In someembodiments, the steps in FIG. 6 may be implemented in program code thatis executed by a processor, for example, the processor in a generalpurpose computer, a mobile device, or a server. In some embodiments,these steps may be implemented by a group of processors. In someembodiments one or more steps shown in FIG. 6 may be omitted orperformed in a different order. Similarly, in some embodiments,additional steps not shown in FIG. 6 may also be performed. The stepsbelow are described with reference to components described above withregard to computing device 201 shown in FIG. 2.

For simplicity, the steps below are described with reference to adrawing application. But the steps below are not limited to such anembodiment, and any combination of the steps can be employed via othertypes of applications and/or devices.

The method 600 begins at step 602 when the processor 202 receives asensor signal associated with a gesture comprising at least a contactwith a surface (e.g., of computing device 201) and a non-contact withthe surface. For example, the processor 202 may execute the drawingapplication. The user may move a stylus (e.g., or a pen or anotherobject) toward the computing device 201, e.g., for performing one ormore drawing operations in the drawing application. The position sensor232 (e.g., a camera) may capture one or more images associated with themovement of the stylus transmit an associated sensor signal to theprocessor 202. In such an embodiment, the images may comprise foregroundfeatures, such as a portion of the stylus, and background features, suchas chairs, tables, walls, desks, etc.

The method 600 continues at step 604 when the processor 202 determines auser interface level based at least in part on the gesture. In someembodiments, the processor 202 is configured to rely on programmingcontained in memory 204, a lookup table, and/or an algorithm todetermine the user interface level. For example, the processor 202 mayuse a lookup table to map a position in real space (that is associatedwith the gesture) to a particular user interface level of the drawingapplication. For example, the processor 202 can map a first position inreal space that is at a far distance from the computing device 101 to afirst user interface level associated with a perspective view of adrawing output in the drawing application. The processor 202 can map asecond position that is at a medium distance from the computing device101 to a second user interface level associated with a close-up and/orcross-sectional view of the drawing output in the drawing application.

In some embodiments, the processor 202 may associate a single gesturewith a user interface level. For example, the processor 202 mayassociate one specific gesture with a first user interface level andanother gesture with a second user interface level. In otherembodiments, the processor 202 may associate a plurality of gestureswith a user interface level. For example, in some embodiments, theprocessor 202 may associate any gestures occurring within a range ofdistances from the computing device 201 (e.g., between 4 inches and 8inches from a surface of the computing device 201) with a single userinterface level.

The method 600 continues at step 606 when the processor 202 determines auser interaction based on the gesture. For example, the processor 202may determine, e.g., based on the three-dimensional start positionand/or end position of the gesture, that the gesture comprises a swipegesture, a hover gesture, or another type of gesture. The processor 202may correlate the particular type and/or location of the gesture to aspecific user interaction associated with the drawing application, suchas activating a line tool, a square shape tool, a paint-brush-sizemodification tool, etc.

The method 600 continues at step 608 when the processor 202 determines afunction based at least in part on the user interface level and/or theuser interaction. In some embodiments, the function comprises outputtingaudio data, video data, and/or information (e.g., a nearby gas station,restaurant, movie theater, police station, or hospital, or a trafficcondition or a speed limit); placing a telephone call; sending a textmessage, SMS message, or e-mail; opening a web browser or accessing awebsite; opening an application; closing an application; performingbackground processing; performing foreground processing; saving a file;opening a file; performing a game function; receiving data; sendingdata; changing an application setting; ordering a product via a network;printing a document; adding an entry to a list; removing an entry fromthe list; recording a sound; playing media content; opening an e-book;performing a calculation; performing a price comparison; checking a bankbalance; and/or any other number and/or configuration of computerfunctions.

In some embodiments, the function comprises outputting, removing,changing, updating, and/or deleting a virtual object in a userinterface. For example, in the drawing application embodiment, thecomputing device 201 may determine a first function associated with thefirst user interface level in response to detecting, e.g., the userhovering a finger or the stylus at a far distance from the computingdevice 201. The first function may comprise, e.g., outputting theperspective view of the drawing. The computing device 201 may furtherdetermine a second function associated with the first user interfacelevel in response to detecting, e.g., another gesture in real space at afar distance from the computing device 201. The second function maycomprise, e.g., drawing on the virtual canvas (e.g., with a virtualpaint brush) along a path defined by the movement of the gesture. Insome embodiments, the computing device 201 may determine a thirdfunction associated with a second user interface level in response todetecting, e.g., the user hovering a finger at a medium distance fromthe computing device 201. The function may comprise, e.g., outputtingthe close-up and/or cross-sectional view of the drawing. In someembodiments, the computing device 201 may further determine a fourthfunction associated with the second user interface level in response todetecting, e.g., another gesture at a medium distance from the computingdevice 201. The fourth function may comprise, e.g., erasing along a pathdefined by the movement gesture. In some embodiments, the computingdevice 201 may determine a fifth function associated with a third userinterface level in response to detecting, e.g., the user contacting thecomputing device 201. The function may comprise, e.g., saving thevirtual image.

The method 600 continues at step 610 when the processor 202 executes thefunction. In some embodiments, the processor 202 may execute thefunction by executing one or more sub-functions. For example, if thefunction comprises saving the virtual image, the processor 202 maycommunicate with one or more servers (e.g., an image storing database)to obtain a privacy key or password for securely storing the virtualimage to a database. The processor 202 may then save the image to thedatabase.

The method 600 continues at step 612 when the processor 202 determinesone or more haptic effects configured to provide substantiallycontinuous haptic feedback throughout the gesture.

In some embodiments, the processor 202 determines the one or more hapticeffects based at least in part on the gesture and/or a position in realspace associated with the gesture. For example, the processor 202 mayaccess a lookup table stored in memory 204 to map positions throughout agesture to specific haptic effects. In such an embodiment, the lookuptable may be preprogrammed in memory and/or programmable by the user.

In some embodiments, the processor 202 determines the one or more hapticeffects based at least in part on a distance between the user and thecomputing device 201. For example, in some embodiments, a user mayperceive a first haptic effect, e.g., comprising a projected solid,liquid, gas, plasma, sound pressure wave, and/or laser beam, as strongerwithin a first distance range (e.g., between 1 mm and 10 cm) from thecomputing device 201 than another type of haptic effect. Thus, in someembodiments, the processor 202 determines the first remote haptic effectin response to detecting that the user is positioned within the firstdistance range (e.g., between 1 mm and 10 cm) from the computing device201.

As another example, in some embodiments, a user may perceive a secondhaptic effect, e.g., comprising an electrostatic-based capacitivecoupling, as stronger within a second distance range (e.g., between 0 mmand 1 mm) from the computing device 201 than another type of hapticeffect. Thus, in some embodiments, the processor 202 determines thesecond haptic effect in response to detecting that the user ispositioned within the second distance range from the computing device201. As still another example, in some embodiments, a user may perceivea third haptic effect, e.g., comprising a local haptic effect, asstronger within a third distance range (e.g., 0 mm and/or contacting thecomputing device 201) from the computing device 201 than another type ofhaptic effect. Thus, in some embodiments, the processor 202 determinesthe third haptic effect in response to detecting that the user ispositioned within the third distance range from the computing device 201(e.g., contacting the computing device).

In some embodiments, the processor 202 determines a plurality of hapticeffects as the user, e.g., approaches the computing device 201 and movesthrough various distance ranges from the computing device 201. Theplurality of haptic effects may comprise any combination of remoteand/or local haptic effects to provide substantially continuous hapticfeedback to the user.

In some embodiments, the processor 202 determines a haptic effectcomprising one or more characteristics (e.g., magnitude, frequency,duration) configured to be perceived by the user as substantially thesame as a previously output haptic effect. For example, in someembodiments, the processor 202 may detect the user gesturing in realspace (e.g., moving a finger toward the computing device 201). In suchan embodiment, the processor 202 may responsively decrease the magnitudeof the haptic effect in response to determining the user is movingcloser to the computing device 201 and/or increase the magnitude of thehaptic effect in response to determining the user is moving farther fromthe computing device 201. This may provide a user with a substantiallyconsistent haptic experience, e.g., regardless of how close or far theuser is to the computing device 201 (e.g., the haptic output device218).

In some embodiments, the processor 202 determines a haptic effectcomprising one or more characteristics (e.g., magnitude, frequency,duration) configured to be perceived by the user as distinct and/ordistinguishable from a previously output haptic effect. For example, insome embodiments, the processor 202 may detect the user gesturing inreal space. In such an embodiment, the processor 202 may responsivelykeep the characteristics of the haptic effect constant. Because theuser, while gesturing, may be moving closer to and/or farther from thesource of the haptic effect, the user may perceive the strength of thehaptic effect as varying (e.g., even though the processor 202 is keepingthe characteristics of the haptic effect constant). This may provideinformation to the user, such as how close the user is to the computingdevice 201.

In some embodiments, the processor 202 determines a haptic effect basedat least in part on the user interface level, the gesture, and/or theuser interaction. For example, the processor 202 may access a lookuptable stored in memory 204 to map a user interface level (e.g.,associated with viewing or drawing on a virtual canvas in the drawingapplication) to specific haptic effects, such as a puff of air oranother remote haptic effect. In some embodiments, the processor 202 isconfigured to determine the haptic effect based on the type, location,duration, and/or other characteristics of the gesture. In someembodiments, the processor 202 may determine the haptic effect based onthe type, location, duration, and/or other characteristics of the userinteraction. For example, in some embodiments, the processor 202determines that the user interaction is configured to erase an image inthe drawing application and, based on this user interaction, determinesan associated haptic effect configured to, e.g., simulate the feeling oferasing a black-board with an eraser.

In some embodiments, the processor 202 determines a haptic effect basedat least in part on the function (e.g., a characteristic associated withthe function). For example, in some embodiments, the processor 202 maydetermine a haptic effect associated with a type of the function. Forinstance, in one embodiment, the processor 202 may determine a hapticeffect comprising a short puff of air if the function comprises openingan application, such as the drawing application. In such an embodiment,the processor 202 may determine a haptic effect comprising three shortpuffs of air if the function comprises closing the application. This mayindicate to the function to the user. In some embodiments, the processor202 determines a haptic effect configured to indicate that the functionhas been executed. For example, the processor 202 may determine a hapticeffect comprising an electrostatic haptic effect once a functioncomprising selecting a new color for a paintbrush tool in the drawingapplication has been executed. In some embodiments, the processor 202 isconfigured to determine different haptic effects based on the result ofexecuting the function. For example, the processor 202 may output adifferent haptic effect if the function was successfully executed thanif the function was not successfully executed.

The method 600 continues at step 614 when the processor 202 outputs theone or more haptic effects. The processor 202 transmits one or morehaptic signals associated with the one or more haptic effects to hapticoutput device 218, which outputs the one or more haptic effect. In someembodiments, the one or more haptic effects are configured to providesubstantially continuous haptic feedback throughout the gesture. Forexample, the processor 202 may transmit a plurality of haptic signalsassociated with a plurality of haptic effects (e.g., remote and/or localhaptic effects). The plurality of haptic effects are configured toprovide substantially continuous haptic feedback throughout the gestureand/or another movement in real space.

Advantages of Position-Based Haptic Effects

There are numerous advantages of position-based haptic effects. Forexample, such systems may provide a more realistic or immersive userexperience. For instance, in some embodiments, such systems may providesubstantially continuous haptic feedback to the user throughout agesture, e.g., as the user approaches and/or contacts a computingdevice. This may more realistically simulate physical phenomena, such asgravity, magnetic fields, electrical fields, wind blowing, resistance toan application of force, and/or heat. For example, in some embodiments,a computing device may execute a virtual apocalypse game in which avirtual nuclear reactor melted down. The computing device may detect auser gesturing toward the computing device and responsively outputhaptic effects configured to, e.g., simulate radiation contamination.For example, the computing device may substantially continuously outputremote haptic effects with increasing amplitude in response to detectingthe user approaching the computing device, culminating in an intensevibration in response to the user contacting the computing device. Thecontinuous, intensifying haptic effects may more realistically simulateapproaching a site contaminated with radiation.

As another example, in some embodiments, a user may interact with avirtual object (e.g., output on a display) by gesturing in a firstlocation in real space associated with a first user interface level. Acomputing device may responsively output haptic feedback configured tosimulate a surface (e.g., a pillow sheet on a pillow) associated with avirtual object. The user may further interact with the virtual object bygesturing in a second location associated with a second user interfacelevel. The computing device may responsively output haptic feedbackconfigured to simulate a different surface (e.g., the inside of thepillow, such as feathers or memory foam) associated with the virtualobject. In some embodiments, this may make the virtual object feel morerealistic and/or three-dimensional. Further, by associating differenthaptic effects with different user interface levels, a user may be ableto perceive a plurality of haptic effects while interacting with asingle virtual object. This may allow for a greater range of hapticexperiences.

In some embodiments, position-based haptic effects may allow the user tomake a state determination (e.g., determine the mode a device is in)without looking at the device. Thus, the user may be able to maintainfocus on other tasks. For example, a computing device may project remotehaptic effects to a user associated with available operations in aprogram or on a user interface. This may allow the user to interact withthe user interface, without having to visually focus on the display.Similarly, a haptic effect may serve as a confirmation that an operationis available, has been completed, or is of a certain level ofimportance.

In some embodiments, position-based haptic effects may allow for moreunique, interactive, and effective user interfaces. For example, in someembodiments, a user may be able to perform a broad range of functionswhile interacting with different user interface levels associated withthe user interface component. Further, in some embodiments, a user maybe able to locate the position of a user interface component by hoveringa body part over the user interface component and activate the userinterface component by moving the body part to another location in realspace (e.g., toward the user interface component). This may enable theuser to interact with the user interface, e.g., without physicallycontacting (e.g., tapping and/or gesturing on) a user interfacecomponent. Physically contacting a user interface component can bechallenging, for example, in a moving vehicle, in which ambient motion,cognitive load, and visual distractions may make it difficult toprecisely tap on a user interface component.

In other embodiments, position-based haptic effects may enablenon-visual interfaces. For example, in some embodiments, a user may beable explore a non-visual user interface with the user's finger bygesturing in real space (e.g., in a location associated with a userinterface level). Upon the user's finger interacting with the locationof an invisible interface component (e.g., a virtual volume switch), thecomputing device may output haptic feedback. This may allow the user toidentify the location of the invisible interface component. Uponidentifying the location of the interface component, the user may beable to interact with (e.g., press on) the interface component, e.g., bymoving the user's finger to another location in real space (e.g.,associated with another user interface level). This may cause thecomputing device to perform a function associated with the invisibleinterface component (e.g., increasing a volume level). The computingdevice may further provide the user with haptic feedback, for example,to confirm receipt of the user input.

GENERAL CONSIDERATIONS

The methods, systems, and devices discussed above are examples. Variousconfigurations may omit, substitute, or add various procedures orcomponents as appropriate. For instance, in alternative configurations,the methods may be performed in an order different from that described,and/or various stages may be added, omitted, and/or combined. Also,features described with respect to certain configurations may becombined in various other configurations. Different aspects and elementsof the configurations may be combined in a similar manner. Also,technology evolves and, thus, many of the elements are examples and donot limit the scope of the disclosure or claims.

Specific details are given in the description to provide a thoroughunderstanding of example configurations (including implementations).However, configurations may be practiced without these specific details.For example, well-known circuits, processes, algorithms, structures, andtechniques have been shown without unnecessary detail in order to avoidobscuring the configurations. This description provides exampleconfigurations only, and does not limit the scope, applicability, orconfigurations of the claims. Rather, the preceding description of theconfigurations will provide those skilled in the art with an enablingdescription for implementing described techniques. Various changes maybe made in the function and arrangement of elements without departingfrom the spirit or scope of the disclosure.

Also, configurations may be described as a process that is depicted as aflow diagram or block diagram. Although each may describe the operationsas a sequential process, many of the operations can be performed inparallel or concurrently. In addition, the order of the operations maybe rearranged. A process may have additional steps not included in thefigure. Furthermore, examples of the methods may be implemented byhardware, software, firmware, middleware, microcode, hardwaredescription languages, or any combination thereof. When implemented insoftware, firmware, middleware, or microcode, the program code or codesegments to perform the necessary tasks may be stored in anon-transitory computer-readable medium such as a storage medium.Processors may perform the described tasks.

Having described several example configurations, various modifications,alternative constructions, and equivalents may be used without departingfrom the spirit of the disclosure. For example, the above elements maybe components of a larger system, wherein other rules may takeprecedence over or otherwise modify the application of the invention.Also, a number of steps may be undertaken before, during, or after theabove elements are considered. Accordingly, the above description doesnot bound the scope of the claims.

The use of “adapted to” or “configured to” herein is meant as open andinclusive language that does not foreclose devices adapted to orconfigured to perform additional tasks or steps. Additionally, the useof “based on” is meant to be open and inclusive, in that a process,step, calculation, or other action “based on” one or more recitedconditions or values may, in practice, be based on additional conditionsor values beyond those recited. Headings, lists, and numbering includedherein are for ease of explanation only and are not meant to belimiting.

Embodiments in accordance with aspects of the present subject matter canbe implemented in digital electronic circuitry, in computer hardware,firmware, software, or in combinations of the preceding. In oneembodiment, a computer may comprise a processor or processors. Theprocessor comprises or has access to a computer-readable medium, such asa random access memory (RAM) coupled to the processor. The processorexecutes computer-executable program instructions stored in memory, suchas executing one or more computer programs including a sensor samplingroutine, selection routines, and other routines to perform the methodsdescribed above.

Such processors may comprise a microprocessor, a digital signalprocessor (DSP), an application-specific integrated circuit (ASIC),field programmable gate arrays (FPGAs), and state machines. Suchprocessors may further comprise programmable electronic devices such asPLCs, programmable interrupt controllers (PICs), programmable logicdevices (PLDs), programmable read-only memories (PROMs), electronicallyprogrammable read-only memories (EPROMs or EEPROMs), or other similardevices.

Such processors may comprise, or may be in communication with, media,for example tangible computer-readable media, that may storeinstructions that, when executed by the processor, can cause theprocessor to perform the steps described herein as carried out, orassisted, by a processor. Embodiments of computer-readable media maycomprise, but are not limited to, all electronic, optical, magnetic, orother storage devices capable of providing a processor, such as theprocessor in a web server, with computer-readable instructions. Otherexamples of media comprise, but are not limited to, a floppy disk,CD-ROM, magnetic disk, memory chip, ROM, RAM, ASIC, configuredprocessor, all optical media, all magnetic tape or other magnetic media,or any other medium from which a computer processor can read. Also,various other devices may comprise computer-readable media, such as arouter, private or public network, or other transmission device. Theprocessor, and the processing, described may be in one or morestructures, and may be dispersed through one or more structures. Theprocessor may comprise code for carrying out one or more of the methods(or parts of methods) described herein.

While the present subject matter has been described in detail withrespect to specific embodiments thereof, it will be appreciated thatthose skilled in the art, upon attaining an understanding of theforegoing may readily produce alterations to, variations of, andequivalents to such embodiments. Accordingly, it should be understoodthat the present disclosure has been presented for purposes of examplerather than limitation, and does not preclude inclusion of suchmodifications, variations and/or additions to the present subject matteras would be readily apparent to one of ordinary skill in the art.

What is claimed:
 1. A system comprising: a sensor configured to detect agesture and transmit a sensor signal associated with the gesture,wherein the gesture comprises at least two positions, a first positionof the at least two positions comprising a distance from a surface and asecond position of the at least two positions comprising a contact withthe surface; and a processor in communication with the sensor, theprocessor configured to: receive the sensor signal from the sensor;determine one or more haptic effects based at least in part on thesensor signal, the one or more haptic effects configured to providesubstantially continuous haptic feedback throughout the gesture;generate one or more haptic signals based at least in part on the one ormore haptic effects; and transmit the one or more haptic signals; and ahaptic output device in communication with the processor, the hapticoutput device configured to receive the one or more haptic signals andoutput the one or more haptic effects.
 2. The system of claim 1, whereinthe gesture comprises a starting position that is remote from thesurface and an ending position that is remote from the surface.
 3. Thesystem of claim 1, wherein the gesture is between the at least twopositions, the gesture starting in the first position and ending in thesecond position.
 4. The system of claim 3, wherein the one or morehaptic effects comprises a remote haptic effect and a local hapticeffect, and wherein the processor is further configured to: cause thehaptic output device to output the remote haptic effect in response todetecting a first interaction at the first position; and cause thehaptic output device to output the local haptic effect in response todetecting a second interaction at the second position.
 5. The system ofclaim 4, wherein the haptic output device comprises a first hapticoutput device positioned in a wearable device configured to be worn by auser and a second haptic output device coupled to the surface; andwherein the processor is further configured to: cause the first hapticoutput device to output the remote haptic effect in response todetecting the first interaction at the first position; and cause thesecond haptic output device to output the local haptic effect inresponse to detecting the second interaction at the second position. 6.The system of claim 4, wherein: the gesture comprises a movement of abody part; the remote haptic effect comprises generating anelectrostatic coupling with the body part or emitting a solid, liquid,gas, plasma, sound pressure wave, or laser beam toward the body part;the local haptic effect comprises a vibration, a texture, a perceivedchange in a coefficient of friction, a surface deformation, or avariation in temperature; and the sensor comprises a camera, a 3Dimaging device, a gyroscope, a global positioning system (GPS), a lightemitting diode (LED), an infrared sensor, or an ultrasonic transducer.7. The system of claim 1, wherein the processor is further configuredto: determine, based at least in part on a position of the at least twopositions, a user interface level from a plurality of available userinterface levels; determine a function associated with the userinterface level; and execute the function.
 8. The system of claim 1,wherein the processor is further configured to: determine a body partbased on a second sensor signal from a second sensor; determine that thebody part comprises a sensitive body part; determine that at least onehaptic effect of the one or more haptic effects is configured to beapplied to the sensitive body part; and determine a characteristic ofthe at least one haptic effect configured to reduce a likelihood ofinjury to the sensitive body part.
 9. A method comprising: receiving asensor signal associated with a gesture from a sensor, wherein thegesture comprises at least two positions, a first position of the atleast two positions comprising a distance from a surface and a secondposition of the at least two positions comprising a contact with thesurface; determining one or more haptic effects based at least in parton the sensor signal, the one or more haptic effects configured toprovide substantially continuous haptic feedback throughout the gesture;generating one or more haptic signals based at least in part on the oneor more haptic effects; and transmitting the one or more haptic signalsto a haptic output device, the haptic output device configured toreceive the one or more haptic signals and output the one or more hapticeffects.
 10. The method of claim 9, wherein the gesture comprises astarting position that is remote from the surface and an ending positionthat is remote from the surface.
 11. The method of claim 9, wherein thegesture is between the at least two positions, the gesture starting inthe first position and ending in the second position.
 12. The method ofclaim 11, wherein the one or more haptic effects comprises a remotehaptic effect and a local haptic effect, and further comprising:outputting the remote haptic effect in response to detecting a firstinteraction at the first position; and outputting the local hapticeffect in response to detecting a second interaction at the secondposition.
 13. The method of claim 12, wherein the haptic output devicecomprises a first haptic output device positioned in a wearable deviceand a second haptic output device coupled to the surface; and furthercomprising: outputting the remote haptic effect via the first hapticoutput device in response to detecting the first interaction at thefirst position; and outputting the local haptic effect via the secondhaptic output device in response to detecting the second interaction atthe second position.
 14. The method of claim 12, wherein: the gesturecomprises a movement of a body part; the remote haptic effect comprisesgenerating an electrostatic coupling with the body part or emitting asolid, liquid, gas, plasma, sound pressure wave, or laser beam towardthe body part; the local haptic effect comprises a vibration, a texture,a perceived change in a coefficient of friction, a surface deformation,or a variation in temperature; and the sensor comprises a camera, a 3Dimaging device, a gyroscope, a global positioning system (GPS), or anultrasonic transducer.
 15. The method of claim 9, further comprising:determining, based at least in part on a position of the at least twopositions, a user interface level from a plurality of available userinterface levels; determining a function associated with the userinterface level; and executing the function.
 16. The method of claim 9,further comprising: determining a body part based on a second sensorsignal from a second sensor; determining that the body part comprises asensitive body part; determining that at least one haptic effect of theone or more haptic effects is configured to be applied to the sensitivebody part; and determining a characteristic of the at least one hapticeffect configured to reduce a likelihood of injury to the sensitive bodypart.
 17. A non-transitory computer readable medium comprising programcode, which when executed by a processor is configured to cause theprocessor to: receive a sensor signal associated with a gesture from asensor, wherein the gesture comprises at least two positions, a firstposition of the at least two positions comprising a distance from asurface and a second position of the at least two positions comprising acontact with the surface; determine one or more haptic effects based atleast in part on the sensor signal, the one or more haptic effectsconfigured to provide substantially continuous haptic feedbackthroughout the gesture; generate one or more haptic signals based atleast in part on the one or more haptic effects; and transmit the one ormore haptic signals to a haptic output device, the haptic output deviceconfigured to receive the one or more haptic signals and output the oneor more haptic effects.
 18. The non-transitory computer readable mediumof claim 17, wherein the gesture comprises a starting position that isremote from the surface and an ending position that is remote from thesurface.
 19. The non-transitory computer readable medium of claim 17,wherein: the gesture is between the at least two positions, the gesturestarting in the first position and ending in the second position; andthe one or more haptic effects comprises a remote haptic effect and alocal haptic effect; and further comprising program code which whenexecuted by the processor is configured to cause the processor to: causethe haptic output device to output the remote haptic effect in responseto detecting a first interaction at the first position; and cause thehaptic output device to output the local haptic effect in response todetecting a second interaction at the second position.
 20. Thenon-transitory computer readable medium of claim 19, wherein the hapticoutput device comprises a first haptic output device positioned in awearable device and a second haptic output device coupled to thesurface; and further comprising program code which when executed by theprocessor is configured to cause the processor to: cause the firsthaptic output device to output the remote haptic effect in response todetecting the first interaction at the first position; and cause thesecond haptic output device to output the local haptic effect inresponse to detecting the second interaction at the second position.